Papers with Multimodal sentiment analysis
Missing Modality meets Meta Sampling (M3S): An Efficient Universal Approach for Multimodal Sentiment Analysis with Missing Modality (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing methods to address missing modalities often assume a particular modality is completely missing due to recording or transmission error. |
| Approach: | They propose a missing modality-based meta-sampling approach for multimodal sentiment analysis with missing modalities . they conduct experiments on IEMOCAP, SIMS and CMU-MOSI datasets . |
| Outcome: | The proposed method significantly improves on existing models with a mixture of missing modalities. |
Improving Multimodal fusion via Mutual Dependency Maximisation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal sentiment analysis is a trending area of research, and multimodal fusion is one of its most active topics. |
| Approach: | They propose to use modality-based penalties to measure dependency between models to improve accuracy. |
| Outcome: | The proposed methods improve accuracy on two well-known sentiment analysis datasets by 4.3 on the proposed models and by-product includes a statistical network which can interpret the high dimensional representations learnt by the model. |
SWAFN: Sentimental Words Aware Fusion Network for Multimodal Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies focus on learning the joint representation of multiple modalities, ignoring useful knowledge contained in language modal. |
| Approach: | They propose to incorporate sentimental words knowledge into the fusion network to guide the learning of joint representation of multimodal features. |
| Outcome: | The proposed method improves the fusion representation of multimodal features on a YouTube and video dataset. |
Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing models for multimodal sentiment analysis are limited in their capacity to be deployed in the real world. |
| Approach: | They propose a model that can dynamically refine erroneous sentiment words by leveraging multimodal sentiment clues. |
| Outcome: | The proposed model surpasses the state-of-the-art models on three datasets. |
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies focus on learning cross-modal dynamics, but neglect to explore optimal solution for unimodal networks. |
| Approach: | They propose a new MSA framework to identify contribution of modalities and reduce impact of noisy information. |
| Outcome: | The proposed model outperforms state-of-the-art methods on publicly available datasets. |
Bridging Modality Gap for Effective Multimodal Sentiment Analysis in Fashion-related Social Media (2025.coling-main)
Copied to clipboard
| Challenge: | Existing sentiment analysis tasks focus on text comprehension, but visual content is important for emotional expression. |
| Approach: | They propose a multimodal framework that integrates information from various modalities for sentiment classification of fashion posts. |
| Outcome: | The proposed framework outperforms existing unimodal and multimodal baselines on a comprehensive dataset and significantly outperformed existing unilmodal and multiple modal frameworks. |
Uncertainty-Calibrated Elastic Alignment for Multimodal Sentiment Analysis with Missing Modalities (2026.findings-acl)
Copied to clipboard
Kang He, Yuzhe Ding, Rao Fu, Yukang Feng, Kaipeng Zhang, Yiming Liu, Fei Li, Chong Teng, Donghong Ji
| Challenge: | Existing methods for multimodal sentiment analysis are often dynamically incomplete. |
| Approach: | They propose a new uncertainty-calibrated elastic alignment framework to address these issues by employing probabilistic imputation to capture cross-modal ambiguity and leverage the estimated uncertainty to drive elastic alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art models in multiple benchmarks and consistently outperformed existing models. |
CTFN: Hierarchical Learning for Multimodal Sentiment Analysis Using Coupled-Translation Fusion Network (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multimodal sentiment analysis require all modalities as input, thus are sensitive to missing modality at predicting time. |
| Approach: | They propose to model bi-direction interplay via couple learning and exploit multiple bi-directional translations to exploit multimodal fusion embeddings. |
| Outcome: | The proposed framework achieves state-of-the-art or often competitive performance on two multimodal benchmarks with extensive ablation studies. |
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. |
| Approach: | They propose a multimodal sentiment knowledge-sharing framework that unifies MSA and ERC tasks from features, labels, and models. |
| Outcome: | The proposed framework achieves consistent improvements on four public benchmark datasets on MOSI, MOSEI, MELD, and IEMOCAP. |
Beyond Static Alignment: Adaptive Arbitration for Semantic Incongruence in Semi-Supervised Multimodal Sentiment Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for semantic incongruence in sentiment analysis are limited by label-limited settings. |
| Approach: | They propose a framework for semi-supervised multimodal sentiment analysis that emphasizes stable cross-modal representations and reliable supervision. |
| Outcome: | The proposed framework outperforms state-of-the-art methods under label-limited settings. |